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(57)Abstract 

PROBLEM TO BE SOLVED: To allow the user to understand 
contents of dynamic image data to be reproduced by means of a 
representative image. 

SOLUTION: A plurality of thumbnails are generated from one set of 
moving image data. In this case, pluralities of frames T1, T2,...Tn of tl 
the dynamic image data are extracted and the plurality of ^ 
thumbnails are generated by using the frames or the one frame T1 
is extracted and the plurality of thumbnails of different sizes are 
generated by using it. In the case of displaying the thumbnail 
generated in this way, the plurality of the thumbnails of each of the 
dynamic image data are displayed through frame-feeding. 
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Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] The thumbnail creation approach characterized by creating two or more thumbnails to said dynamic-image 
data when taking out one frame of dynamic-image data as still picture data and considering as a thumbnail. 
[Claim 2] The thumbnail creation approach which is the thumbnail creation approach according to claim 1 t and is 
characterized by creating two or more thumbnails from which the amount of data differs to one frame of said 
dynamic-image data. 

[Claim 3] The thumbnail creation approach characterized by being the creation approach of a thumbnail according to 
claim 1, detecting a scene change in said dynamic-image data, and creating a thumbnail automatically from the 
frame corresponding to the image immediately after a scene change. 

[Claim 4] The thumbnail creation approach which is the creation approach of a thumbnail according to claim 1, and 
is characterized by setting a fixed time interval and creating a thumbnail automatically to said dynamic-image data. 
[Claim 5] The thumbnail method of presentation characterized by being the thumbnail method of presentation which 
displays two or more thumbnails, determining the display sequence of a thumbnail based on the image of a thumbnail 
and the information on correlation of dynamic-image data which are recorded on the management information of a 
thumbnail, and performing the coma delivery display of said thumbnail by said display sequence. 

[Claim 6] The thumbnail method of presentation which is the thumbnail method of presentation according to claim 5, 
and is characterized by repeating a coma delivery display and performing it from the thumbnail displayed first again 
after finishing displaying the thumbnail to dynamic-image data. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the creation approach and the method of presentation of the 

thumbnail which is a cutback image for indicating the content of dynamic-image data by simple. 

[0002] 

[Description of the Prior Art] The description acquired by treating dynamic-image data with a disk-like storage is 
being able to perform random access to dynamic-image data easily. However, it is closely related to an AV 
equipment or the facility of AV record format how it can display on a user in what kind of content the point which 
carries out random access is. This problem is so serious that the amount of data recorded especially turns into a 
large quantity. 

[0003] In case a user chooses desired dynamic-image data from dynamic-image data with the amount of data of the 
large quantity which has more than one, it is important to perform a quick and brief content display. 
[0004] For example, in the case of the digital still camera (still picture camera by JPEG compression) with spread 
remarkable recently, the method of performing the menu display using a thumbnail is often used so that a user may 
tend to do selection playback of the still picture photographed by the large quantity. In the case of a digital still 
camera, the still picture data of one sheet are recorded as one file, and the image data for thumbnails is also 
simultaneously recorded on storages (a memory card, floppy disk, etc.) as one file. At this time, the approach of the 
relation between this image and a cutback image (thumbnail) preparing a regulation (for example, the same prefix or 
a suffix being attached) in a file name, and recognizing is often used. Moreover, since the number of the images to 
treat was one in the case of the digital still camera, it was enough, when creating and recording the image for 
thumbnails after creating and recording this image. 
[0005] 

[Problem(s) to be Solved by the Invention] In the example of the digital camera explained in the above-mentioned 
conventional technique, the object of the image data to deal with is a static image. When a cutback image expresses 
still picture data, it is possible to express the content of said still picture data by the cutback image of one sheet, 
but when the data for retrieval are dynamic-image data, it is difficult for one representation cutback image to 
express the content. 

[0006] For example, when the dynamic-image data for several minutes are recorded on one file, in order for the 
image of one sheet to express the content, a user has to select the representation image from dynamic-image data, 
or it must make by edit. 

[0007] This invention solves the above-mentioned technical problem which serves as trouble in identification of the 
dynamic-image data currently recorded on the disk-like storage explained in the above-mentioned conventional 
technique. 
[0008] 

[Means for Solving the Problem] In order to express the content of dynamic-image data promptly and briefly, 
invention which starts claim 1 in order to solve the above-mentioned technical problem takes out one frame of 
dynamic-image data as still picture data, considers as a thumbnail, and is characterized by creating two or more 
thumbnails to said dynamic-image data. 

[0009] Moreover, invention concerning claim 2 is characterized by creating two or more thumbnails from which the 
amount of data differs to one frame of said dynamic-image data, in order to increase the class of method of 
presentation of a thumbnail. 

[0010] Moreover, in order that invention concerning claim 3 may carry out the content display of dynamic-image 
data to a detail, the scene change in dynamic-image data is detected as an approach of creating two or more 
thumbnail images, and it is characterized by carrying out automatic creation of the image immediately after a scene 
change as a thumbnail. 

[0011] Moreover, invention concerning claim 4 is characterized by vacating a fixed time interval to dynamic-image 
data in creation of a thumbnail as an approach of creating two or more thumbnail images for carrying out the 
content display of dynamic-image data to a detail, and creating a thumbnail automatically. 

[0012] Moreover, invention concerning claim 5 determines the display sequence of a thumbnail based on the image 
of a thumbnail and the information on correlation of dynamic-image data which are recorded on the management 
information of a thumbnail as an approach of making in agreement the display sequence of the content of dynamic- 
image data, and a thumbnail, and displaying it, and is characterized by performing the coma delivery display of said 
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thumbnail by said display sequence. 

[0013] Moreover, in order to help for a user to understand the content of dynamic-image data by repeating and 
displaying the thumbnail of dynamic-image data, after invention concerning claim 6 finishes displaying the thumbnail 
to dynamic-image data, it is characterized by repeating a coma delivery display and performing it from the thumbnail 
displayed first again. 
[0014] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this invention is explained, referring to a 
drawing. In addition, in the following explanation, a "thumbnail" shall be the static image to which the display size 
was reduced from the subject-copy image, and shall be a cutback static image related with the frame which exists 
in dynamic-image data by management information. 

[0015] Drawin g 1 is the block diagram of the dynamic-image data logging and the regenerative apparatus in which 
creation, record, and a display of the thumbnail which is one of the operation gestalten of this invention are possible. 

[0016] First, each component of the dynamic-image data logging and the regenerative apparatus of drawing 1 
explains the actuation performed at the time of creation, record, and a display of a thumbnail. 

[0017] (User I/F section 100) The user I/F section 100 controls telling record / playback control section about the 
directions from input units, such as a keyboard by the user, and remote control, etc. 

[0018] (Record / playback control section 110) Record / playback control section 1 10 is a part which controls the 
whole system. Control instruction is sent to each part of the dynamic-image data-logging section 120, the dynamic- 
image data playback section 130, and the control data Management Department 140 based on the directions from a 
user sent from the user I/F section 1 00. 

[0019] (Dynamic-image data-logging section 120) The dynamic-image data-logging section 120 has the video-signal 
input section 121, the video-signal coding section 122, and a frame memory 123. 

[0020] - In order to change into dynamic-image data the video signal inputted from video-signal input section 121 
tuner, the camera, etc., send to the video-signal coding section 1 22. 

[0021] In creation of a thumbnail, the video-signal input section 121 takes out the frame which creates a thumbnail 
from a video signal, and once stores in the frame memory 123 constituted by semiconductor memory etc. After 
sending all the inputted video signals to the video-signal coding section 122, the frame for thumbnails stored in the 
frame memory 123 is sent to the video-signal coding section 122. 

[0022] Moreover, in case the video-signal input section 121 takes out the frame for thumbnail creation from a video 
signal, it sends a frame number (to the frame of what position in a video signal does the frame correspond?) to the 
control data Management Department 140. 

[0023] - Do compression and coding of the video signal sent from the video-signal coding section 122 video-signal 
input section 121 using coding methods, such as MPEG 2 (13818 to ISO/IEC2 reference), and change into dynamic- 
image data. The changed dynamic-image data are recorded on a storage 200 through the below-mentioned file 
system section 1 50. 

[0024] In creation of a thumbnail, the frame for thumbnail images sent from the video-signal input section 121 is 
reduced, compression and coding of are done using coding methods, such as MPEG 2 and JPEG (ISO/IEC10918 
reference), and the image data of a thumbnail is created. 

[0025] The image data of the created thumbnail is memorized by the storage section 141 which the control data 
Management Department 140 manages. Moreover, the video-signal coding section 122 sends the size information 
(magnitude of an image, amount of data of an image, etc.) acquired in the case of this coding to the control data 
Management Department 140. 

[0026] - Frame memory 1 23 frame memory 1 23 consists of semiconductor memory etc., and stores temporarily the 
frame for thumbnail images taken out from the video signal. 

[0027] (Dynamic-image data playback section 130) The dynamic-image data playback section 130 has the dynamic- 
image data input section 131, the dynamic-image data decode section 132, and the video-signal output section 133. 
[0028] - In response to the instruction of record / playback control section 110, the dynamic-image data input 
section 131 dynamic-image data input section 131 reads the image data of the thumbnail currently stored in the 
storage section 141 based on the thumbnail management information managed at the control data Management 
Department 140 so that it may mention later, and it sends it to the dynamic-image data decode section 132. 
[0029] - The dynamic-image data decode section 132 dynamic-image data decode section 132 decodes the image 
data of the thumbnail encoded in the form of [ which has been sent from the dynamic-image data input section 
131 ] MPEG 2, JPEG, etc. The image data of the decoded thumbnail is sent to the video-signal output section 133. 
[0030] - The video-signal output section 133 video-signal output section 133 processes compounding two or more 
thumbnails on one screen based on thumbnail management information etc. to the image data of the thumbnail sent 
from the dynamic-image data decode section 1 32, and outputs it to a display. 

[0031] (Control data Management Department 140) The control data Management Department 140 creates 
thumbnail management information, records those management information on the storage section 141, and 
manages a thumbnail. 

[0032] Thumbnail management information is the record positional information of the frame number in dynamic- 
image data, and the image data of a thumbnail, and is created based on the image data of the frame number sent 
from the dynamic-image data-logging section 1 20, and a thumbnail. 

[0033] Moreover, the information on the coding formats (MPEG, JPEG, etc.) of the image size of a thumbnail or a 
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thumbnail image is also recorded on thumbnail management information. 

[0034] Whenever the control data Management Department 140 creates or updates the thumbnail management 
information or the thumbnail image data currently recorded on the storage section 141. it writes in the storage 200 
which mentions thumbnail management information or thumbnail image data later. 

[0035] - The storage section 141 storage section 141 records the image data of the thumbnail created in the 
thumbnail management information created at the control data Management Department 140, and the dynamic- 
image data-logging section 1 20. 

[0036] (File system section 150) The file system section 150 enables access to the dynamic-image data memorized 
with the digital signal on the storage 200 in units, such as a file or a sector, and is random per the file or sector, and 
performs control which makes access possible. 

[0037] (Storage 200) The management information of dynamic-image data and a thumbnail and the image data of a 
thumbnail are recorded on the storage 200. As a storage 200 used here, an optical disk, HDD of the storage of the 
non-volatile in which search by several 1 00 or less mses is possible at least etc. are desirable to the data currently 
recorded. 

[0038] Then, the creation procedure of a thumbnail using dynamic-image data logging and the regenerative 
apparatus of drawing 1 and the method of presentation are explained. 

[0039] (1) The creation procedure video-signal input section 121 of a thumbnail copies two or more frames to 
ejection from the inputted video signal, and copies two or more frames for the ta ken-out frame to a frame memory 
123 for thumbnail creation. At this time, the information acquired in case a frame is copied from video signals, such 
as information on the playback location in the inside of the video signal of the copied frame, is sent to the control 
data Management Department 140. 

[0040] The frame copied to the frame memory 1 23 chooses the time of there being no video signal which the video- 
signal coding section 122 should process at its own discretion, and is sent to the video-signal coding section 122. 
[0041] In the video-signal coding section 122, a series of frames sent from the video-signal input section 121 are 
reduced to the size of a thumbnail, and it encodes. 

[0042] The image data of the information acquired in the case of coding, such as image size of a thumbnail, and the 
encoded thumbnail is sent to the control data Management Department 140. 

[0043] At the control data Management Department 140, the management information of a thumbnail is created 
based on the information acquired in case frames, such as playback positional information in the inside of the video 
signal of the frame sent from the video-signal input section 121, are copied, and the information acquired in the 
case of coding, such as image size of the thumbnail sent from the video-signal coding section 1 22. 
[0044] The image data of the thumbnail management information which the control data Management Department 
140 created, and a thumbnail is recorded on the storage section 141. 

[0045] The image data of the thumbnail management information currently recorded on the storage section 140 and 
a thumbnail is recorded on a storage 200 through the file system section 1 50 for every updating. 
[0046] (2) The thumbnail display procedure control data Management Department 140 will send the dynamic-image 
data of the thumbnail currently recorded on the storage section 141 based on the management information of a 
thumbnail to the dynamic-image data decode section 133 via the dynamic-image data input section 131 of the 
dynamic-image data playback section 1 30, if the display instruction of the thumbnail from record / playback control 
section 1 1 0 is received. 

[0047] The dynamic-image data decode section 133 decodes the received thumbnail image data, and sends it to the 
video-signal output section 133. In the video-signal output section 133, it compounds so that two or more images of 
two or more thumbnails sent from the dynamic-image data decode section 1 33 can be displayed on the one display 
screen based on the thumbnail management information which the control data Management Department 140 
creates and manages, and it outputs to a display. 

[0048] As it explained above, dynamic-image data logging and the regenerative apparatus of drawing 1 perform 
creation and a display of a thumbnail. 

[0049] In addition, in the above-mentioned explanation, how two or more frames for thumbnail creation are decided 
does not specify from the inputted video signal. Two or more frames are because it can choose by the approach of 
arbitration. 

[0050] as an approach of determining two or more frames and creating a thumbnail, a scene change takes place in a 
dynamic image, for example — ** — the approach of it being alike and creating a thumbnail for the frame 
corresponding to the image after a scene change using ejection and it — or how to create a thumbnail for a frame 
using ejection and it for every fixed time amount can be considered. Below, these two approaches are explained 
briefly. 

[0051] By the approach of creating a thumbnail, when a scene change occurs, detection of scene change generating 
is performed first. For example, when the root mean square of YUV of each pixel of an adjacent frame and the 
difference on RGB color matching space exceeds the threshold decided beforehand. Or when there is voice data 
reproduced synchronizing with dynamic-image data and the number of dynamic-image data and the voice data 
reproduced by coincidence changes. Or the time of detecting the changing point from information other than video 
signals, such as an electronic program guide (EPG), can be considered as the event of a scene change occurring. In 
this invention, if it does in this way and a scene change is detected, a thumbnail will be created for the frame 
corresponding to an image immediately after a scene change occurs using ejection and it. 

[0052] For example, the case where a scene change is detected based on the root mean square of the difference of 
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a pixel is considered. The frame of the video signal inputted into the video-signal input section 121 of drawing 1 is 
recorded on a frame buffer 123. Let this frame be Frame A. Next, when the frame inputted into the video-signal 
input section 121 is used as Frame B, the difference of the magnitude of the vector on a color space called YUV 
and RGB which each pixel of Frame A has, and the vector which the pixel corresponding to each pixel of the frame 
A of Frame B has is taken, and it asks for a root mean square. When the root mean square exceeds the value of the 
threshold decided beforehand, after storing Frame B in the frame buffer 123 as a subject-copy image for thumbnails 
as what the scene change generated and completing coding of a video signal, it sends to the image coding section 
122, it encodes, and changes into thumbnail image data. 

[0053] Thus, by the thumbnail creation approach of this invention, whenever a scene change occurs, a thumbnail 
can be created. 

[0054] Next, how to create a thumbnail for every fixed time amount is explained. By this approach, automatic 
creation of the thumbnail corresponding to the frame for every fixed time interval is carried out to the frame located 
in a line with the time series of the playback time amount in dynamic-image data. When the video signal with which 
the frame of fixed number of sheets follows fixed time amount inputs into the video-signal input section 121, the 
frame inputted every fixed number of sheets is stored in a frame buffer 123 as a frame for thumbnails. It is 
compressed and encoded in the video-signal coding section 122, and it changes into the image data of a thumbnail 
image. 

[0055] Moreover, when data with the information about the display time of day of the frame at the time of playback 
like PTS of an MPEG 2 system stream (ISO / 13818 to IEC1 reference) are inputted as a video signal, the video- 
signal input section 121 chooses the frame displayed with a fixed time interval from the information, and stores it in 
a frame buffer 1 23. 

[0056] Thus, it is also possible to create a thumbnail for every fixed time amount by the thumbnail creation 
approach of this invention. 

[0057] In addition, in the explanation so far, the thumbnail of two or more sheets has been obtained from one 
dynamic-image data to one dynamic-image data by creating each of the taken-out frame to ejection and a 
thumbnail for two or more frames. However, this invention may not be restricted to this, but may take out one frame 
from one dynamic-image data, and may create the thumbnail of two or more sheets from the frame, in this case — 
for example, as shown in drawing 2 , it can have image size which is different in the thumbnail of two or more sheets 
created from one frame. 

[0058] Drawin g 2 shows the response relation between a thumbnail image and dynamic-image data. It has expressed 
with the frame train as which dynamic-image data are displayed along with a time-axis in this drawing. Moreover, T1 
to Tn is a frame which the thumbnail in dynamic-image data shows. 

[0059] As mentioned above, in the thumbnail creation approach of this invention, the thumbnail of two or more 
sheets is created from two or more frames of a series of dynamic-image data. Or the thumbnail of image size which 
is different like drawin g 2 also from one frame accomplishes two or more works. The created thumbnail is carried 
out by thumbnail management information in correlation with the frame which the information on the image data- 
logging location of a thumbnail and the thumbnail in a series of dynamic-image data show so that a thumbnail image 
may point out the frame in dynamic-image data. 

[0060] Then, an example of the approach of displaying the thumbnail of two or more sheets is explained, referring to 
drawing 3 . 

[0061] The thumbnail of two or more sheets is displayed on the display screen. Display number of sheets displays a 
1 76x1 12-pixel thumbnail on a 720x480-pixel screen, and supposing there is no superposition of thumbnails, it will 
become a maximum of 1 6 sheets. As for the thumbnail on a screen, each supports one dynamic-image data. That is, 
one in the thumbnail of two or more sheets matched with each dynamic-image data is displayed on the screen 
about two or more dynamic-image data. 

[0062] The thumbnail displayed on a screen is updated one after another so that the thumbnail of two or more 
sheets matched with each dynamic-image data may be altogether displayed in order by coma delivery display. After 
the display of all the thumbnails of each dynamic-image data finishes, from the thumbnail displayed first again, a 
coma delivery display is repeated and is performed. 

[0063] If a user chooses one thumbnail from two or more thumbnails currently displayed on the screen, the dynamic 
image corresponding to the selected thumbnail will be chosen, and playback will be started. 
[0064] Thus, according to the thumbnail method of presentation of this invention, the content of two or more 
dynamic-image data can be simultaneously displayed by coma delivery display, and it can be made easy that a user 
chooses one in two or more dynamic-image data. 

[0065] Moreover, it is also possible to change the order of a display of the thumbnail of two or more sheets matched 
with each dynamic-image data. Below, modification of the order of a thumbnail display is explained. 
[0066] Thumbnail management information is the positional information on the time-axis by which the record 
positional information of thumbnail image data and the frame in the dynamic-image data corresponding to the 
thumbnail are reproduced, and the display sequence of a thumbnail is determined by the record positional 
information of thumbnail image data, or the playback positional information of the frame in the dynamic-image data 
on a time-axis. That is, it is not necessary to rearrange the image data of a thumbnail into display sequence, and the 
display sequence of a thumbnail is changed by rearrangement of the information on the playback location on 
rearrangement of the information which shows the location which recorded the image data of the thumbnail 
currently recorded on thumbnail management information, or the time-axis of the frame in the dynamic-image data 
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corresponding to a thumbnail. 

[0067] In drawin g 4 , TA4 and TB1 to TB3 is created from the thumbnail TA 1 corresponding to PB3 from the frame 
PB1 of the frame PA 1 of the dynamic-image data A to PA4, and the dynamic-image data B. If the dynamic-image 
data B are inserted among PA2 and PA3 of the dynamic-image data A here, display sequence of the thumbnail of 
the newly made dynamic-image data C must be set to PA1, PA2, PB1, PB2, PB3, PA4, and PA5. At this time, the 
thumbnail of the dynamic-image data C is not newly created, but the record positional information of the thumbnail 
image data to "the image of a thumbnail which the dynamic-image data A and the dynamic-image data B have", or 
the playback positional information of the frame in the dynamic-image data on a time-axis is rearranged according 
to the list of the frame of the dynamic-image data C. 
[0068] 

[Effect of the Invention] As explained above, in this invention, the effectiveness that it becomes easy for a user to 
understand the content of the dynamic-image data is acquired by displaying the thumbnail on a screen in the 
content display of dynamic-image data by having the thumbnail which is the cutback image of two or more sheets 
created from two or more frames belonging to the dynamic-image data. 

[0069] Moreover, by having the thumbnail from which the amount of data differs to the frame belonging to dynamic- 
image data, the burden of the equipment by modification of the image size of a thumbnail or resolution and a user is 
reduced, and in a display pattern, it carries out and the effectiveness that it becomes easy an increase and for a 
user to understand the content of the dynamic-image data is acquired. 

[0070] Moreover, whenever a thumbnail detects a scene change, the effectiveness of displaying on a detail by a 
series of thumbnails which had the content of dynamic-image data created, and reducing the burden of the user in 
connection with thumbnail creation of a large quantity is acquired by being created automatically. 
[0071] Moreover, the effectiveness of displaying on a detail by a series of thumbnails which had the content of 
dynamic-image data created, and reducing the burden of the user in connection with thumbnail creation of a large 
quantity is acquired by a thumbnail's setting a fixed time interval and creating it automatically. 
[0072] Moreover, by determining the display sequence of a thumbnail based on the image of a thumbnail and the 
information on correlation of dynamic-image data which are recorded on the management information of a 
thumbnail, by [ of said thumbnail ] indicating by coma delivery, the display sequence of the content of dynamic- 
image data and a thumbnail is made in agreement, it displays, and the effectiveness that it becomes easy for a user 
to understand the content of dynamic-image data is acquired by said display sequence. 

[0073] Moreover, after finishing displaying the thumbnail to dynamic-image data, by repeating a coma delivery 
display and performing it from the thumbnail displayed first again, the content of repeat dynamic-image data is 
shown to a user, and the effectiveness that it becomes easy for a user to understand the content of the dynamic- 
image data is acquired. 



[Translation done.] 



http://wvm4jpdl.jpo.gojp/cgi-bin/tran_web_cgi_ejje 



2004/08/06 




1/1 ^— V 



* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram of the dynamic-image data logging and the regenerative apparatus in which creation, 
record, and a display of a thumbnail are possible 

[Drawin g 2] Drawing showing the response relation between a thumbnail image and dynamic-image data 
[Drawing 3] Drawing showing the notation of a thumbnail 

[Drawing 4] Drawing showing the modification approach of the display sequence of a thumbnail 

[Description of Notations] 

100 User I/F Section 

1 10 Record / Playback Control Section 

120 Dynamic-Image Data-Logging Section 

121 Video-Signal Input Section 

122 Video-Signal Coding Section 

123 Frame Memory 

130 Dynamic-Image Data Playback Section 

131 Dynamic-Image Data Input Section 

132 Dynamic-Image Data Decode Section 

133 Video-Signal Output Section 

140 Control Data Management Department 

141 Storage Section 

1 50 File System Section 



[Translation done.] 
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